Display control device, display control system, display control method, and display control program

ABSTRACT

A display control device includes a video data acquisition unit that acquires video data from a camera that captures a video image in a moving direction of a vehicle, a vehicle motion detection unit that detects motion information containing a steering angle of the vehicle, an expected path line generation unit that generates a main expected path line extending along a planned moving path in accordance with the steering angle from a center part of a width of the vehicle, a superimposed video generation unit that generates superimposed data, the superimposed data being video data where the main expected path line is superimposed on the video data, and a display control unit that transmits the superimposed data to a display unit so as to display a video related to the superimposed data on the display unit.

CROSS REFERENCE TO RELATED APPLICATION

The present application is a Continuation of U.S. patent applicationSer. No. 16/714,508, filed on Dec. 13, 2019, which is a BypassContinuation of PCT/JP2018/000367 filed on Jan. 10, 2018, which is basedupon and claims the benefit of priority from Japanese Patent ApplicationNo. 2017-118402, filed on Jun. 16, 2017, and Japanese Patent ApplicationNo. 2017-118401, filed on Jun. 16, 2017, the disclosure of which areincorporated herein in its entirety by reference.

BACKGROUND

The present invention relates to a display control device, a displaycontrol system, a display control method, and a display control program.

A technique that, when a vehicle is moving backward, superimposes aguide line indicating a predicted path of backward motion on a rearwardvideo image captured using a rear camera mounted at the back of thevehicle and displays this video image on a display unit such as amonitor to easily and accurately guide the vehicle to a parking spacehas become popular recently.

For example, the vehicle-mounted imaging device disclosed in JapaneseUnexamined Patent Application Publication No. 2011-211432 takes a videoshowing the back of a vehicle at the time of parking the vehicle, forexample, and displays the video image showing the back of the vehicle onan in-vehicle monitor, and further superimposes a predicted path in themoving direction of the vehicle on this video image showing the back ofthe vehicle.

SUMMARY

FIG. 24 shows an example of the related art according to PatentLiterature 1. A video image 40 shown in this example is a video imagecaptured using a rear camera mounted at the back of a vehicle, on whicha guide line 70 indicating a predicted path of the vehicle's backwardmotion is superimposed. A driver is moving a vehicle 10 to a parkingspace 33 that includes parking lines 30 and 31. The guide line 70corresponds to the width of the vehicle 10 and is drawn along anintended path of the vehicle. The parking line 30 and the guide line 70overlap.

As described above, in the related art, a guide line superimposed on arearward video image overlaps a parking line in some cases. In suchcases, it is difficult for a driver to find out the parking line in thedisplayed video image. Thus, there are cases where it is difficult for adriver to grasp the positional relationship between a vehicle and aparking space.

The present embodiment has been accomplished to solve the above problemsand an object of the present invention is thus to provide a displaycontrol device, a display control system, a display control method, anda display control program that reduce difficulty of visually recognizinga parking line displayed on a display unit.

A display control device according to an embodiment includes a videodata acquisition unit configured to acquire video data from a camerathat captures a video image in a moving direction of a vehicle, avehicle motion detection unit configured to detect motion informationcontaining a steering angle of the vehicle, an expected path linegeneration unit configured to generate a first expected path lineextending along a planned moving path in accordance with the steeringangle from a center part along a width of the vehicle and a thirdexpected path line including a pair of lines extending along a plannedmoving path of the vehicle from both ends of the vehicle in the widthdirection, a parking line recognition unit configured to recognize aparking line from the video data, a detection unit configured to detecta nearby object around the vehicle, a determination unit configured todetermine to superimpose the first expected path line on the video datawhen a distance between the nearby object detected by the detection unitand the parking line is larger than a predetermined value, and theparking line and the third expected path line are to be displayed inclose proximity to each other, a superimposed video generation unitconfigured to generate superimposed data, the superimposed data beingvideo data where the first expected path line or the third expected pathline is superimposed on the video data in accordance with adetermination by the determination unit, and a display control unitconfigured to transmit the superimposed data to a display unit so as todisplay a video related to the superimposed data on the display unit.

A display control method according to an embodiment includes acquiringvideo data from a camera that captures a video image in a movingdirection of a vehicle, detecting motion information containing asteering angle of the vehicle, generating a first expected path lineextending along a planned moving path in accordance with the steeringangle from a center part along a width of the vehicle and a thirdexpected path line including a pair of lines extending along a plannedmoving path of the vehicle from both ends of the vehicle in the widthdirection, recognizing a parking line from the video data, detecting anearby object around the vehicle, determining to superimpose the firstexpected path line on the video data when a distance between the nearbyobject detected by the detection unit and the parking line is largerthan a predetermined value, and the parking line and the third expectedpath line are to be displayed in close proximity to each other,generating superimposed data, the superimposed data being video datawhere the first expected path line or the third expected path line issuperimposed on the video data in accordance with a determination, andtransmitting the superimposed data to a display unit so as to display avideo related to the superimposed data on the display unit.

A display control program according to an embodiment causes a computerto execute acquiring video data from a camera that captures a videoimage in a moving direction of a vehicle, detecting motion informationcontaining a steering angle of the vehicle, generating a first expectedpath line extending along a planned moving path in accordance with thesteering angle from a center part along a width of the vehicle and athird expected path line including a pair of lines extending along aplanned moving path of the vehicle from both ends of the vehicle in thewidth direction, recognizing a parking line from the video data,detecting a nearby object around the vehicle, determining to superimposethe first expected path line on the video data when a distance betweenthe nearby object detected by the detection unit and the parking line islarger than a predetermined value, and the parking line and the thirdexpected path line are to be displayed in close proximity to each other,generating superimposed data, the superimposed data being video datawhere the first expected path line or the third expected path line issuperimposed on the video data in accordance with a determination, andtransmitting the superimposed data to a display unit so as to display avideo related to the superimposed data on the display unit.

According to the embodiment, it is possible to provide a display controldevice, a display control system, a display control method, and adisplay control program that reduce difficulty of visually recognizing aparking line.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a functional block diagram of a display control system 100according to an embodiment.

FIG. 2 is a top view showing an example of a state where a vehicle 10 onwhich the display control system 100 is mounted is moving to a parkingspace.

FIG. 3 is a view showing an example of a video image where video datacaptured using a camera in the vehicle 10 is displayed on a display unit220.

FIG. 4A is a view showing an example of a video image where a mainexpected path line is superimposed on video data in the display controlsystem 100 according to the first embodiment.

FIG. 4B is a view showing an example of a video image where a mainexpected path line is superimposed on video data in the display controlsystem 100 according to the first embodiment.

FIG. 5 is a view showing a flowchart in a display control device 200according to the first embodiment.

FIG. 6A is a view showing an example of a video image where a mainexpected path line and a vehicle width guide line are superimposed onvideo data in the display control system 100 according to a secondembodiment.

FIG. 6B is a view showing an example of a video image where a mainexpected path line and a vehicle width guide line are superimposed onvideo data in the display control system 100 according to the secondembodiment.

FIG. 7A is a view showing an example of a video image where a mainexpected path line and a vehicle width guide line are superimposed onvideo data in the display control system 100 according to a thirdembodiment.

FIG. 7B is a view showing an example of a video image where a mainexpected path line and a vehicle width guide line are superimposed onvideo data in the display control system 100 according to the thirdembodiment.

FIG. 8A is a view showing another display example of a main expectedpath line in the display control system 100 according to a fourthembodiment.

FIG. 8B is a view showing another display example of a main expectedpath line in the display control system 100 according to the fourthembodiment.

FIG. 9 is a functional block diagram of a display control system 1100according to a fifth embodiment.

FIG. 10 is a view showing an example of a video image where a firstexpected path line is superimposed on video data in the display controlsystem 1100 according to the fifth embodiment.

FIG. 11A is a view showing an example of a video image where a secondexpected path line is superimposed on video data in the display controlsystem 1100 according to the fifth embodiment.

FIG. 11B is a view showing an example of a video image where a secondexpected path line is superimposed on video data in the display controlsystem 1100 according to the fifth embodiment.

FIG. 12 is a view showing a flowchart in a display control device 1200according to the fifth embodiment.

FIG. 13A is a view showing an example of a video image in the displaycontrol system 1100 according to a sixth embodiment.

FIG. 13B is a view showing an example of a video image in the displaycontrol system 1100 according to the sixth embodiment.

FIG. 14 is a view showing a flowchart in the display control device 1200according to the sixth embodiment.

FIG. 15 is a functional block diagram of a display control system 1101according to a seventh embodiment.

FIG. 16 is a top view showing an example of a state where a vehicle 10on which the display control system 1101 according to the seventhembodiment is mounted is moving to a parking space.

FIG. 17A is a view showing an example of a video image in a displaycontrol system 1101 according to the seventh embodiment.

FIG. 17B is a view showing an example of a video image in the displaycontrol system 1101 according to the seventh embodiment.

FIG. 18 is a view showing a flowchart in a display control device 1201according to the seventh embodiment.

FIG. 19A is a view showing an example of a video image in a displaycontrol system 1101 according to an eighth embodiment.

FIG. 19B is a view showing an example of a video image in the displaycontrol system 1101 according to the eighth embodiment.

FIG. 20 is a view showing a flowchart in a display control device 1201according to the eighth embodiment.

FIG. 21 is a view showing an example of a video image where a firstexpected path line and a vehicle width guide line are superimposed onvideo data.

FIG. 22A is a view showing an example of a video image where a firstexpected path line and a second expected path line with hightransparency are superimposed on video data.

FIG. 22B is a view showing an example of a video image where a firstexpected path line and a second expected path line with hightransparency are superimposed on video data.

FIG. 23 is a view showing a video image illustrating a variation of afirst expected path line.

FIG. 24 is a view showing an example of a rearward video image in arelated art.

DETAILED DESCRIPTION Embodiment

Embodiments are described hereinafter with reference to the drawings.FIG. 1 is a functional block diagram of a display control system 100according to an embodiment. The functional block of the display controlsystem 100 shown in FIG. 1 shows elements common to first to fourthembodiments described hereinbelow.

The display control system 100 is a system that, when a driver is movinga vehicle to a parking space or the like, superimposes an expected pathline indicating a planned moving direction on a video image captured bya camera 210 that is mounted on the front or back of the vehicle andtakes a video of a vehicle moving direction, and displays this videoimage on a display unit 220. The display control system 100 may beimplemented as one function of a car navigation system, a dashboardcamera or the like which can be retrofitted to a vehicle, for example.Further, a display control device 200 may be implemented as one functionof a system incorporated into a vehicle when assembling the vehicle in afactory, for example. Furthermore, the display control device 200 may beimplemented by combining a system incorporated into a vehicle whenassembling the vehicle in a factory and a device that can be retrofittedto a vehicle, for example. The display control system 100 includes thedisplay control device 200 and at least one of the camera 210 and thedisplay unit 220.

The display control device 200 acquires video data, which is data of avideo captured by the camera 210, superimposes an expected path line onthis data and displays the data on the display unit 220. The displaycontrol device 200 may be implemented as a device in various forms suchas a computer device that operates when a processor executes a programstored in a memory. The processor may be a microprocessor, an MPU (MicroProcessing Unit), or a CPU (Central Processing Unit), for example. Thememory is a volatile memory or a nonvolatile memory, and it may be acombination of a volatile memory and a nonvolatile memory. The processorexecutes one or a plurality of programs including a group ofinstructions for causing a computer to perform processing according toan embodiment. The display control device 200 is connected with a CAN(Controller Area Network) or the like, which is an in-car network.Further, the display control device 200 is connected with the camera 210and the display unit 220. The CAN or the like means that it may be anin-car network such as MOST (Media Oriented Systems Transport) orEthernet, not limited to the CAN.

Further, the program can be stored and provided to the computer usingany type of non-transitory computer readable medium. The non-transitorycomputer readable medium includes any type of tangible storage medium.Examples of the non-transitory computer readable medium include magneticstorage media (such as floppy disks, magnetic tapes, hard disk drives,etc.), optical magnetic storage media (e.g. magneto-optical disks),CD-ROM (Read Only Memory), CD-R, CD-R/W, and semiconductor memories(such as mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flashROM, RAM (Random Access Memory), etc.). The program may be provided to acomputer using any type of transitory computer readable medium. Examplesof the transitory computer readable medium include electric signals,optical signals, and electromagnetic waves. The transitory computerreadable medium can provide the program to a computer via a wiredcommunication line such as an electric wire or optical fiber or awireless communication line.

The display control device 200 is described hereinafter in detail. Thedisplay control device 200 includes a vehicle motion detection unit 300,an expected path line generation unit 310, a video data acquisition unit320, a superimposed video generation unit 350, and a display controlunit 360.

The vehicle motion detection unit 300 receives motion information, whichis information about the motion of a vehicle from a CAN or the like,which is an in-car network. The motion information is informationcontaining some or all of information indicating to which range thetransmission of a vehicle is set, information indicating the steeringangle of a vehicle, and information indicating the traveling speed of avehicle. Specifically, the vehicle motion detection unit 300 detects apredetermined signal among signals transmitted from a CAN or the like,and transmits this signal to the expected path line generation unit 310or the display control unit 360. By receiving the motion information,the vehicle motion detection unit 300 can detect that the transmissionis set to the reverse range, for example. Further, the vehicle motiondetection unit 300 can detect that the transmission is set to thelow-speed driving range, for example. Furthermore, the vehicle motiondetection unit 300 can detect information indicating the steering angleof a vehicle and monitor the angle of the steering wheel of the vehicle.The vehicle motion detection unit 300 can detect information indicatingthe traveling speed of a vehicle and monitor the traveling speed of thevehicle.

The expected path line generation unit 310 receives the motioninformation transmitted from the vehicle motion detection unit 300.Then, the expected path line generation unit 310 generates an expectedpath line based on the motion information transmitted from the vehiclemotion detection unit 300. The expected path line is line informationindicating the planned moving path of a vehicle by being superimposed onvideo data captured by the camera 210. The expected path line generationunit 310 calculates an expected path based on information indicating thesteering angle transmitted from the vehicle motion detection unit 300,for example. The expected path line generation unit 310 generates anexpected path line from the calculated expected path. An arbitraryexisting technique may be applied to the generation of the expected pathline by the expected path line generation unit 310. For example, theexpected path line generation unit 310 calculates one point or two ormore points as expected path points. Then, the expected path linegeneration unit 310 generates information of the expected path line thatpasses through the expected path points by a predetermined method. Anexample of the expected path line is line information indicating linesthat extend the vehicle width when the vehicle moves in a straight line.Another example of the expected path line is line information associatedwith the steering angle of the vehicle. The expected path linegeneration unit 310 transmits information about the generated expectedpath line to the superimposed video generation unit 350. The expectedpath line generation unit 310 generates and transmits information aboutone or a plurality of expected path lines among the above-describedexpected path lines.

The video data acquisition unit 320 acquires the video data generated inthe camera 210. The video data acquisition unit 320 acquiresuncompressed digital video data or analog video data from the camera210. The video data acquisition unit transmits the video data to thesuperimposed video generation unit 350 as uncompressed digital videodata or compressed video data in a format such as H.264 or H.265, forexample. Further, the video data acquisition unit 320 may generate videodata in a compressed video format such as MPEG (Moving Picture ExpertsGroup) 2-TS (Transport Stream) or AVI (Audio Video Interleave). Thevideo data acquisition unit 320 transmits the video data to thesuperimposed video generation unit 350.

The superimposed video generation unit 350 receives the informationtransmitted from the expected path line generation unit 310 and thevideo data transmitted from the video data acquisition unit 320. Thesuperimposed video generation unit 350 then superimposes the expectedpath line on the video data.

The display control unit 360 receives the superimposed data transmittedfrom the superimposed video generation unit 350. The display controlunit 360 transmits the received superimposed data to the display unit220. The display control unit 360 may transmit the superimposed data tothe display unit 220 or stop transmitting the superimposed data inresponse to an instruction from a display instruction unit, which is notshown, for example. The display control unit 360 may monitor the vehiclemotion detection unit 300, and transmit the video data or thesuperimposed data to the display unit 220 or stop transmitting thosedata in accordance with information detected by the vehicle motiondetection unit 300. Further, the display control unit 360 may connect tothe vehicle motion detection unit 300 and acquire vehicle motioninformation.

The display control unit 360 may process the received superimposed dataso as to change the viewing angle of a video image to be displayed onthe display unit. Further, the display control unit 360 may process thereceived superimposed data so as to perform distortion correction of avideo image to be displayed on the display unit. Furthermore, thedisplay control unit 360 may process the received superimposed data soas to change a video image to be displayed on the display unit to avideo image viewed from the top of the vehicle. The display control unit360 may process the received superimposed data so as to change a videoimage to be displayed on the display unit in a way not limited theabove.

The camera 210 generates video data that shows a frontward or rearwardvideo of a vehicle. The camera 210 may be placed on the front of avehicle. Alternatively, the camera 210 may be placed on the back of avehicle. Further, the camera 210 may be placed on each of the front of avehicle and the back of the vehicle. The camera 210 transmits thegenerated video data to the video data acquisition unit 320 in thedisplay control device 200.

The display unit 220 displays the video data or superimposed datareceived from the display control unit 360 in the display control device200. The display unit 220 is a display device such as a liquid crystaldisplay device, an organic EL (organic electro-luminescence) displaydevice, or a head-up display, for example. The display unit 220 isplaced at a position that can be viewed by a driver while handling avehicle. The position that can be viewed by a driver while handling avehicle is a position on a center console, on a dashboard, on a meterpanel, on a rear-view mirror, near a steering wheel, on a windshield orthe like, for example.

A vehicle on which the display control system 100 is mounted isdescribed hereinafter with reference to FIG. 2. FIG. 2 is a top viewshowing an example of a state where a vehicle 10 on which the displaycontrol system 100 is mounted is moving to a parking space. The vehicle10 includes the display control device 200 at an arbitrary position. Thevehicle 10 also includes a forward camera 210F and a rearward camera210R. The vehicle 10 further includes the display unit 220 at a positionthat can be viewed by a driver while handling the vehicle. In FIG. 2,the vehicle 10 is moving backward toward a parking space 33. The parkingspace 33 includes parking lines 30 to 32. The driver is trying to movethe vehicle 10 to a position surrounded by the parking lines 30 to 32.In this case, the display control system 100 acquires data of a videocaptured by the rearward camera 210R.

The video data captured by the camera of the vehicle 10 is describedhereinafter. FIG. 3 is a view showing an example of a video image wherevideo data captured by a camera in the vehicle 10 is displayed on thedisplay unit 220. A video image 40 contains a part of the vehicle 10 andthe parking lines 30 to 32. The lower side of the video image 40 shows aposition closer to the vehicle 10, and the upper side of the video image40 shows a position farther from the vehicle 10.

FIG. 3 contains x-y coordinates. In FIG. 3, the x-coordinate indicatesthe horizontal direction of the video image 40. The rightward directionof FIG. 3 is the positive direction of the x-axis. In FIG. 3, they-coordinate indicates the vertical direction of the video image 40. Theupward direction of FIG. 3 is the positive direction of the y-axis. Thex-y coordinates in FIG. 3 are shown for the convenience of explainingthe positional relationship of elements. The x-y coordinates shown inFIGS. 4 to 7 are shown for the same purpose as the x-y coordinates inFIG. 3.

First Embodiment

In light of the foregoing, a first embodiment is described hereinafterwith reference to FIG. 4. In a display control system 100 according to afirst embodiment, the expected path line generation unit 310 generates amain expected path line that extends along a planned moving path inaccordance with the steering angle from a center part along the width ofthe vehicle 10. Note that, in the present disclosure, the main expectedpath line is referred to also as a first expected path line.

FIG. 4 is a view showing an example of a video image where a mainexpected path line is superimposed on video data in the display controlsystem 100 according to the first embodiment. A video image 40 in FIG.4A contains a main expected path line 50 in addition to the video imagedescribed in FIG. 3. The main expected path line 50 extends along aplanned moving path in accordance with the steering angle from a centerpart 11 along the width of the vehicle 10. In other words, the mainexpected path line 50 is line information indicating an expected drivingpath of the vehicle 10 in accordance with the steering angle of thevehicle 10. The main expected path line 50 is a curved line inaccordance with the steering angle of the vehicle 10 except when thevehicle 10 is moving in a straight line. The curvature of the mainexpected path line 50 is larger as the steering angle of the vehicle 10is larger.

A main expected path line 51 superimposed on the video image 40 shown inFIG. 4B is a straight line. The main expected path line 51 extendslinearly from a start point 50 s coinciding with the center part 11 ofthe vehicle 10 along a planned moving path in accordance with thesteering angle. Thus, the main expected path line 51 connects the startpoint 50 s and an end point 50 e by a straight line. The main expectedpath line 51 draws a straight line parallel to the y-axis when thevehicle 10 is moving in a straight line. The angle between the mainexpected path line 51 and the y-axis is larger as the steering angle ofthe vehicle 10 is larger.

The start point 50 s is contained in the center part 11 of the vehicle10. The center part 11 is a center region that occupies about one-thirdof the width of the vehicle 10 displayed on the video image 40. Thestart point 50 s may coincide with a center point 11 c in the centerpart 11. The end point 50 e is a point on a curved line extending fromthe start point 50 s along the planned moving path in accordance withthe steering angle of the vehicle.

The end point 50 e may indicate a position where the actual distancefrom the vehicle 10 corresponds to a predetermined value. In the videoimage 40, a length 50 y in the y-axis direction from the start point 50s to the end point 50 e corresponds to a distance of 3 meters, forexample, from the center part 11 of the vehicle 10. In this case, thelength 50 y up to the end point 50 e in the y-axis direction can befixed. Further, the position of the end point 50 e in the x-axisdirection varies depending on the steering angle of the vehicle 10. Asthe steering angle of the vehicle 10 is larger, the absolute value of alength 50 x in the x-axis direction increases. When the steeringdirection of the vehicle 10 is right in the video image 40, the mainexpected path line 50 extends from the center part 11 in the y-axispositive direction and the x-axis positive direction. Likewise, when thesteering direction of the vehicle 10 is left in the video image 40, themain expected path line 50 extends from the center part 11 in the y-axispositive direction and the x-axis negative direction.

Note that, the length 50 y up to the end point 50 e in the y-axisdirection may remain unfixed, and it may change to draw a circle or anellipse around the start point 50 s. The length 50 y up to the end point50 e in the y-axis direction may remain unfixed, and it may change insuch a way that the length of the main expected path line 50 issubstantially the same even when the steering angle changes.

The main expected path line 50 is not necessarily a solid line as shownin FIG. 4, and it may be a dashed line or a dotted line. The mainexpected path line 50 is not necessarily equally bold, and its width mayvary from the start point 50 s to the end point 50 e.

As described above, the main expected path line 50 or the main expectedpath line 51 extends in the moving direction of the vehicle 10 from thecenter part 11 of the vehicle 10. Thus, the possibility that the parkingline and the main expected path line overlap in the y-axis direction ofthe video image 40 is reduced when the driver moves the vehicle to theparking space. Therefore, the display control system 100 according tothe first embodiment can reduce difficulty of visually recognizing theparking line.

A process of the display control device 200 according to the firstembodiment is described hereinafter with reference to FIG. 5. FIG. 5 isa view showing a flowchart in the display control device 200 accordingto this embodiment.

First, the vehicle motion detection unit 300 detects information aboutthe setting of the transmission of the vehicle 10 from a CAN or thelike, which is an in-car network (Step S100).

Next, the display control unit 360 acquires the information about thesetting of the transmission of the vehicle 10 detected by the vehiclemotion detection unit 300, and determines whether the vehicle 10 is tomove backward or not (Step S101). When the vehicle 10 is not to movebackward, which is, when the setting of the transmission of the vehicle10 is not a reverse range (No in Step S101), the display control unit360 determines whether an instruction to end video display is receivedor not (Step S106). When an instruction to end video display is received(Yes in Step S106), the display control unit 360 ends the process. Onthe other hand, when an instruction to end video display is not received(No in Step S106), the process returns to Step S100, and the vehiclemotion detection unit 300 detects information about the setting of thetransmission of the vehicle 10 again.

On the other hand, when the vehicle 10 is moving backward, which is,when the setting of the transmission of the vehicle 10 is a reverserange (Yes in Step S101), the display control unit 360 performsprocessing of transmitting superimposed data to the display unit 220.Specifically, the video data acquisition unit 320 acquires video datagenerated by the camera 210 and transmits it to the superimposed videogeneration unit 350 (Step S102). Further, the expected path linegeneration unit 310 generates a main expected path line and transmits itto the superimposed video generation unit 350 (Step S103).

Next, the superimposed video generation unit 350 generates superimposeddata, which is video data where the expected path line transmitted fromthe expected path line generation unit 310 is superimposed on the videodata transmitted from the video data acquisition unit 320, and transmitsthis superimposed data to the display control unit 360 (Step S104).

Then, the display control unit 360 transmits the superimposed datatransmitted from the superimposed video generation unit 350 to thedisplay unit 220, and displays the data thereon (Step S105). The displaycontrol unit 360 then determines whether an instruction to end videodisplay is received or not (Step S106). When an instruction to end videodisplay is received (Yes in Step S106), the display control unit 360ends the process.

On the other hand, when an instruction to end video display is notreceived (No in Step S106), the display control device 200 returns toStep S100, and the vehicle motion detection unit 300 detects informationabout the setting of the transmission of the vehicle 10 again.

The order of performing Step S102 and S103 is arbitrary. Thus, Step S102and S103 may be interchanged. Alternatively, Step S102 and S103 may beperformed in parallel.

The display control system 100 according to the first embodimentperforms the above-described process and can thereby reduce difficultyof visually recognizing the parking line displayed on the display unit.

Second Embodiment

A second embodiment is described hereinafter with reference to FIG. 6. Adisplay control system 100 according to the second embodiment isdifferent from the display control system 100 according to the firstembodiment in that the expected path line generation unit 310 generatesa vehicle width guide line in addition to the main expected path line.The vehicle width guide line is a straight line that intersects with themain expected path line 50 or the expected path line 51, extends alongthe width of the vehicle 10, and coincides with the width of the vehicle10. The vehicle width guide line is not limited to a straight line, andit may be a curved line as long as it can appropriately indicate thewidth of the vehicle 10.

FIG. 6 is a view showing an example of a video image where a mainexpected path line and a vehicle width guide line are superimposed onvideo data in the display control system 100 according to the secondembodiment. In the video image 40 shown in FIG. 6A, vehicle width guidelines 66 to 68 are superimposed on the main expected path line 50described in the first embodiment.

The vehicle width guide lines 66 to 68 are line information extending inthe horizontal direction of the video image 40, which is the directionparallel to the x-axis. The vehicle width guide lines 66 to 68 mayindicate a predetermined distance from the end of the vehicle 10. Thevehicle width guide line 66 may indicate a distance of 0.4 meter fromthe rear end of the vehicle 10, for example. The vehicle width guideline 67 may indicate a distance of 1.0 meter from the rear end of thevehicle 10, for example. The vehicle width guide line 68 may indicate adistance of 2.0 meters from the rear end of the vehicle 10, for example.In this case, the vehicle width guide lines 66 to 68 may be lines thatare not displaced in the y-axis direction in the video image 40. Thevehicle width guide lines 66 to 68 do not necessarily extend in thehorizontal direction of the video image 40, and they may extend in theorthogonal direction at the intersection with the main expected pathline 50.

The length of the vehicle width guide lines 66 to 68 coincides with thewidth of the vehicle 10. For example, a left end 66L of the vehiclewidth guide line 66 indicates one end along the width of the vehicle 10at a distance of 0.4 meter from the rear end of the vehicle 10. On theother hand, a right end 66R of the vehicle width guide line 66 indicatesthe other end along the width of the vehicle 10 at a distance of 0.4meter from the rear end of the vehicle 10. Thus, the vehicle width guideline 66 indicates that the vehicle 10 passes through an area between theleft end 66L and the right end 66R at a distance of 0.4 meter from therear end of the vehicle 10, for example. Likewise, the vehicle widthguide line 67 indicates that the vehicle 10 passes through an areabetween a left end 67L and a right end 67R at a distance of 1.0 meterfrom the rear end of the vehicle 10, for example. Further, the vehiclewidth guide line 68 indicates that the vehicle 10 passes through an areabetween a left end 68L and a right end 68R at a distance of 2.0 metersfrom the rear end of the vehicle 10, for example. The vehicle widthguide lines 66 to 68 are displaced in the x-axis direction in the videoimage 40 in accordance with the steering angle of the vehicle 10.

In the video image 40 shown in FIG. 6B, vehicle width guide lines 66 to68 are superimposed on the main expected path line 51 described in thefirst embodiment. The vehicle width guide lines 66 to 68 in the videoimage 40 shown in FIG. 6B also indicate a predetermined distance in they-axis direction, just like the example shown in FIG. 6A. The vehiclewidth guide lines 66 to 68 are displaced in the x-axis direction in thevideo image 40 in accordance with the steering angle of the vehicle 10.

In this configuration, the possibility that the parking line and themain expected path line overlap in the y-axis direction of the videoimage 40 is reduced. Further, the display control device 200 accordingto the second embodiment superimposes vehicle width guide lines and canthereby reduce difficulty of visually recognizing the parking line, andalso displays the distance between the vehicle 10 and the parking lineand can thereby accurately display the positional relationship betweenthe width of the vehicle 10 and the parking space.

Third Embodiment

A third embodiment is described hereinafter with reference to FIG. 7. Adisplay control system 100 according to the third embodiment isdifferent from the display control system 100 according to the firstembodiment in that the expected path line generation unit 310 generatesa sub-expected path line in addition to the main expected path line. Thesub-expected path line is a pair of expected path lines extending fromboth ends of the vehicle 10 in the width direction along a plannedmoving path of the vehicle 10 and having higher transparency than themain expected path line on video data. Note that, in the presentdisclosure, the sub-expected path line is referred to also as a secondexpected path line.

FIG. 7 is a view showing an example of a video image where a mainexpected path line and a sub-expected path line are superimposed onvideo data in the display control system 100 according to the thirdembodiment. In the video image 40 shown in FIG. 7, sub-expected pathlines 52 and 53 are superimposed on the main expected path line 50described in the first embodiment.

The sub-expected path lines 52 and 53 are a pair of line informationassociated with the steering angle of the vehicle 10. The sub-expectedpath lines 52 and 53 are line information indicating an expected drivingpath of the vehicle 10 in accordance with the steering angle of thevehicle 10. The sub-expected path line 52 extends along a planned movingpath in accordance with the steering angle from an end 12 along thewidth of the vehicle 10. Likewise, the sub-expected path line 53 extendsfrom an end 13 along the width of the vehicle 10 along a planned movingpath in accordance with the steering angle. Thus, the sub-expected pathlines 52 and 53 extend in the moving direction of the vehicle 10 fromthe positions corresponding to the width of the vehicle 10. Thesub-expected path lines 52 and 53 are curved lines in accordance withthe steering angle of the vehicle 10 except when the vehicle 10 ismoving in a straight line. The curvature of the sub-expected path lines52 and 53 is larger as the steering angle of the vehicle 10 is larger.

The sub-expected path lines 52 and 53 have predetermined transparency.The predetermined transparency is 50% to 90%, for example. In a partwhere the sub-expected path lines 52 and 53 overlap the video data inthe superimposed data generated by the superimposed video generationunit 350, the sub-expected path lines 52 and 53 are semi-transparent.

Further, the sub-expected path lines 52 and 53 have higher transparencythan the main expected path line 50. For example, when the transparencyof the sub-expected path lines 52 and 53 is 50%, the transparency of themain expected path line 50 is 0%. A difference between the transparencyof the sub-expected path lines 52 and 53 and the transparency of themain expected path line 50 is preferably 50% or more.

In this configuration, the possibility that the parking line and themain expected path line overlap in the y-axis direction of the videoimage 40 is reduced. Further, the display control device 200 accordingto the third embodiment superimposes the sub-expected path lines 52 and53 and can thereby reduce difficulty of visually recognizing the parkingline displayed on the display unit and also accurately display thepositional relationship between the width of the vehicle 10 and theparking space.

FIG. 7B is described hereinafter. In the video image 40 shown in FIG. 7Bsub-expected path lines 54 and 55 are superimposed on the main expectedpath line 50.

The sub-expected path lines 54 and 55 are a pair of line informationcorresponding to the width of the vehicle 10 and extending from the endsalong the width of the vehicle 10 when the vehicle 10 moves in astraight line. The sub-expected path line 54 extends in the straightmoving direction of the vehicle 10 from an end 12 along the width of thevehicle 10. Likewise, the sub-expected path line 55 extends in thestraight moving direction of the vehicle 10 from an end 13 along thewidth of the vehicle 10. Thus, the sub-expected path lines 54 and 55extend in the moving direction of the vehicle 10 from the positionscorresponding to the width of the vehicle 10. The sub-expected pathlines 54 and 55 do not vary depending on the steering angle of thevehicle 10. On the other hand, the main expected path line 50 variesdepending on the steering angle. Therefore, a distance 54 g between anend point 50 e of the main expected path line 50 and an end point 54 eof the sub-expected path line 54 varies depending on the steering angle.Specifically, the distance 54 g is longer as the steering angle of thevehicle 10 is large to the right. Likewise, a distance 55 g between theend point 50 e of the main expected path line 50 and an end point 55 eof the sub-expected path line 55 is longer as the steering angle of thevehicle 10 is large to the left.

In this configuration, the possibility that the parking line and themain expected path line overlap in the y-axis direction of the videoimage 40 is reduced. Further, the display control device 200 accordingto the third embodiment superimposes the sub-expected path lines 54 and55 and can thereby reduce difficulty of visually recognizing the parkingline displayed on the display unit and also accurately display thepositional relationship between the width of the vehicle 10 and theparking space and the information about the steering angle.

Fourth Embodiment

A fourth embodiment is described hereinafter with reference to FIG. 8.

A display control system 100 according to the fourth embodiment shows avariation on the display form of a main expected path line. FIG. 8 is aview showing another display example of a main expected path line in thedisplay control system 100 according to the fourth embodiment.

FIG. 8A is a view showing a main expected path line 56 in the displaycontrol system 100 according to the fourth embodiment. The main expectedpath line 56 is a band-like display defined by a start point 56 a, astart point 56 b, an end point 56 c and an end point 56 d. The startpoint 56 a and the start point 56 b are in the center part 11 along thewidth of the vehicle 10. The position of the center part 11 in thisembodiment is a center region that occupies about one-third of the widthof the vehicle 10 displayed on the video image 40.

The end point 56 c is on a line extending from the start point 56 a inthe moving direction of the vehicle 10. The end point 56 d is on a lineextending from the start point 56 b in the moving direction of thevehicle 10. Further, the positions of the end point 56 c and the endpoint 56 d in the y-axis direction are such that the distance from thevehicle 10 is a predetermined value. The start point 56 a is locatedbetween a center point 11 c along the width of the vehicle 10 and a leftend 12 along the width of the vehicle 10 in the video image 40. Thestart point 56 b is located between the center point 11 c along thewidth of the vehicle 10 and a right end 13 along the width of thevehicle 10 in the video image 40. The start point 56 a and the startpoint 56 b are preferably symmetric along the y-axis with respect to thecenter point 11 c. The distance between the start point 56 a and thestart point 56 b may be longer than the distance between the end point56 c and the end point 56 d. A line connecting the start point 56 a andthe end point 56 c and a line connecting the start point 56 b and theend point 56 d may be curved as shown in FIG. 8A. Alternatively, a lineconnecting the start point 56 a and the end point 56 c and a lineconnecting the start point 56 b and the end point 56 d may be straight.

FIG. 8B is a view showing main expected path lines 57 and 58 in thedisplay control system 100 according to the fourth embodiment. A startpoint 57 s and a start point 58 s are placed in the center part 11, onboth sides of the center point 11 c along the width of the vehicle 10.

An end point 57 e is on a line extending from the start point 57 c inthe moving direction of the vehicle 10. An end point 58 e is on a lineextending from the start point 58 s in the moving direction of thevehicle 10. Further, the positions of the end point 57 e and the endpoint 58 e in the y-axis direction are such that the distance from thevehicle 10 is a predetermined value.

The start point 57 s is located between the center part 11 along thewidth of the vehicle 10 and the left end 12 along the width of thevehicle 10 in the video image 40. The start point 58 s is locatedbetween the center part 11 along the width of the vehicle 10 and theright end 13 along the width of the vehicle 10 in the video image 40.The start point 57 s and the start point 58 s are preferably symmetricalong the y-axis with respect to the center point 11 c. The distancebetween the start point 57 s and the start point 58 s may be longer thanthe distance between the end point 57 e and the end point 58 e. A lineconnecting the start point 57 s and the end point 57 e and a lineconnecting the start point 58 s and the end point 58 e may be curved asshown in FIG. 8B. Alternatively, a line connecting the start point 57 sand the end point 57 e and a line connecting the start point 58 s andthe end point 58 e may be straight.

The two main expected path lines 57 and 58 shown in FIG. 8B are placedin the center part 11, on both sides of the center point 11 c along thewidth of the vehicle 10. Note that, however, three or more main expectedpath lines may be placed in the center part 11.

The main expected path lines 56 to 58 extend in the moving direction ofthe vehicle 10 from the center part 11 of the vehicle 10. Thus, thepossibility that the parking line and the main expected path lineoverlap is reduced when the driver moves the vehicle to the parkingspace. Further, the main expected path lines 56 to 58 are displayed insuch a way that they can be accurately visually recognized by a driver.The display control system 100 according to the fourth embodiment canthereby reduce difficulty of visually recognizing the parking linedisplayed on the display unit.

The above-described embodiments may be applied to the case where thevehicle 10 moves forward to the parking space 33. In this case, thedisplay control device 200 can superimpose the expected path line ondata of a video captured by the forward camera 210F of the vehicle 10.In this case, in Step S100, the display control device 200 can detectthat the vehicle 10 is moving forward at a speed slower than apredetermined traveling speed, for example. Alternatively, in Step S100,the display control device 200 may start a first display mode inresponse to an instruction from a driver, which is not shown.

Fifth Embodiment

A fifth embodiment of the present invention is described hereinafterwith reference to the drawings. FIG. 9 is a functional block diagram ofa display control system 1100 according to the fifth embodiment.

The display control system 1100 is a system that, when a driver ismoving a vehicle to a parking space or the like, superimposes anexpected path line indicating a planned moving direction on a videoimage captured by a camera 210 that is mounted on the front or back ofthe vehicle and takes a video of a vehicle moving direction, anddisplays this video image on a display unit 220. The display controlsystem 1100 may be implemented as one function of a car navigationsystem, a dashboard camera or the like which can be retrofitted to avehicle, for example. Further, a display control device 1200 may beimplemented as one function of a system incorporated into a vehicle whenassembling the vehicle in a factory, for example. Furthermore, thedisplay control device 1200 may be implemented by combining a systemincorporated into a vehicle when assembling the vehicle in a factory anda device that can be retrofitted to a vehicle, for example. The displaycontrol system 1100 includes the display control device 1200 and atleast one of the camera 210 and the display unit 220.

The display control device 1200 acquires video data, which is data of avideo captured by the camera 210, superimposes an expected path line onthis data and displays the data on the display unit 220. The displaycontrol device 1200 may be implemented as a device in various forms suchas a computer device that operates when a processor executes a programstored in a memory.

The display control device 1200 is described hereinafter in detail. Thedisplay control device 1200 includes a vehicle motion detection unit1300, an expected path line generation unit 1310, a video dataacquisition unit 1320, a parking line recognition unit 1330, adetermination unit 1340, a superimposed video generation unit 1350, anda display control unit 1360.

The vehicle motion detection unit 1300 receives motion information,which is information about the motion of a vehicle from a CAN or thelike, which is an in-car network. The motion information is informationcontaining some or all of information indicating to which range thetransmission of a vehicle is set, information indicating the steeringangle of a vehicle, and information indicating the traveling speed of avehicle. Specifically, the vehicle motion detection unit 1300 can detecta predetermined signal among signals transmitted from a CAN or the like,and transmit this signal to the expected path line generation unit 1310,the determination unit 1340 or the display control unit 1360. Byreceiving the motion information, the vehicle motion detection unit 1300can detect that the transmission is set to the reverse range, forexample. Further, the vehicle motion detection unit 1300 can detect thatthe transmission is set to the low-speed driving range, for example.Furthermore, the vehicle motion detection unit 1300 can detectinformation indicating the steering angle of a vehicle and monitor theangle of the steering wheel of the vehicle. The vehicle motion detectionunit 1300 can detect information indicating the traveling speed of avehicle and monitor the traveling speed of the vehicle.

The expected path line generation unit 1310 receives the motioninformation transmitted from the vehicle motion detection unit 1300.Then, the expected path line generation unit 1310 generates an expectedpath line based on the motion information transmitted from the vehiclemotion detection unit 1300. The expected path line is line informationindicating the planned moving path of a vehicle by being superimposed onvideo data captured by the camera 210. The expected path line generationunit 1310 calculates an expected path based on information indicatingthe steering angle transmitted from the vehicle motion detection unit1300, for example. The expected path line generation unit 1310 generatesan expected path line from the calculated expected path. An arbitraryexisting technique may be applied to the generation of the expected pathline by the expected path line generation unit 1310. For example, theexpected path line generation unit 1310 calculates one point or two ormore points as expected path points. Then, the expected path linegeneration unit 1310 generates information of the expected path linethat passes through the expected path points by a predetermined method.An example of the expected path line is line information indicatinglines that extend the vehicle width when the vehicle moves in a straightline. Another example of the expected path line is line informationassociated with the steering angle of the vehicle. The expected pathline generation unit 1310 transmits information about the generatedexpected path line to the determination unit 1340 and the superimposedvideo generation unit 1350. The expected path line generation unit 1310generates and transmits information about one or a plurality of expectedpath lines among the above-described expected path lines.

The video data acquisition unit 1320 acquires the video data generatedin the camera 210. The video data acquisition unit 1320 acquiresuncompressed digital video data or analog video data from the camera210. The video data acquisition unit transmits the video data to thesuperimposed video generation unit 1350 as uncompressed digital videodata or compressed video data in a format such as H.264 or H.265, forexample. Further, the video data acquisition unit 1320 may generatevideo data in a compressed video format such as MPEG (Moving PictureExperts Group) 2-TS (Transport Stream) or AVI (Audio Video Interleave).The video data acquisition unit 1320 transmits the video data to theparking line recognition unit 1330 and the superimposed video generationunit 1350.

The parking line recognition unit 1330 recognizes parking linesdisplayed in a parking space from the video data transmitted from thevideo data acquisition unit 1320. For example, the parking linerecognition unit 1330 calculates a luminance gradient of each pixel ofthe video data and recognizes parking lines by using a predeterminedtechnique such as edge detection. A technique of calculating a luminancegradient of each pixel of video data and recognizing a target object isa known technique. Thus, the detailed description thereof is omitted.The parking line recognition unit 1330 recognizes parking lines andthereby generates information about whether parking lines exist in thevideo data and the positions of parking lines in the video data. Theparking line recognition unit 1330 transmits the generated informationto the determination unit 1340. Further, when the parking linerecognition unit 1330 does not recognize any parking line, it maytransmit information indicating that no parking line is contained in thevideo data to the determination unit 1340.

The determination unit 1340 receives information transmitted from thevehicle motion detection unit 1300, information transmitted from theexpected path line generation unit 1310, or information transmitted fromthe parking line recognition unit 1330, and determines a display form ofthe expected path line to be displayed on the display unit 220. Thedetermination unit 1340 transmits information about this determinationto the superimposed video generation unit 1350.

The determination unit 1340 can determine a display form of the expectedpath line based on the information transmitted from the vehicle motiondetection unit 1300, for example. Further, the determination unit 1340calculates the distance between the parking space and the expected pathline from the information about the expected path line transmitted fromthe expected path line generation unit 1310 and the information aboutthe parking line transmitted from the parking line recognition unit1330, for example. Then, the determination unit 1340 can determine adisplay form of the expected path line based on the calculation result.Note that, when determining a display form of the expected path line,the determination unit 1340 may give an instruction on the form of theexpected path line to be generated to the expected path line generationunit 1310. In this case, the expected path line generation unit 1310 maygenerate the expected path line based on the instruction from thedetermination unit 1340.

The determination unit 1340 may determine a display form by determiningwhich expected path line is to be superimposed when there are aplurality of expected path lines, for example. Further, thedetermination unit 1340 may determine a display form by determiningwhich of a plurality of expected path lines is to be displayed in a lessvisible form when there are a plurality of expected path lines, forexample.

Note that a less visible display form means that it is less noticeablethan another display in a video image displayed on the display unit 220.Specifically, a less visible display form is to reduce the width of anexpected path line to be superimposed on video data, for example.Further, a less visible display form is to increase the transparency ofan expected path line to be superimposed on video data, for example.Furthermore, a less visible display form may include not displaying anexpected path line superimposed on video data, for example.

The superimposed video generation unit 1350 receives informationtransmitted from the expected path line generation unit 1310,information transmitted from the video data acquisition unit 1320, andinformation transmitted from the determination unit 1340. Thesuperimposed video generation unit 1350 superimposes the expected pathline on the video data based on the information transmitted from thedetermination unit. Specifically, when the superimposed video generationunit 1350 superimposes information about the expected path linetransmitted from the expected path line generation unit 1310 on thevideo data transmitted from the video data acquisition unit 1320, it canchange the display form of the expected path line in accordance with theinformation transmitted from the determination unit 1340.

For example, when the determination unit 1340 has transmittedinformation about determination indicating that the expected path lineis to be displayed in a less visible form, the superimposed videogeneration unit 1350 superimposes the expected path line in a lessvisible form on the video data. Alternatively, when the determinationunit 1340 has transmitted information about determination indicatingthat the expected path line is to be displayed not in a less visibleform, the superimposed video generation unit 1350 superimposes theexpected path line not in a less visible form on the video data. Whenthe expected path line generation unit 1310 transmits information abouta plurality of expected path lines, the superimposed video generationunit 1350 superimposes the plurality of expected path lines on the videodata in accordance with information about the display form of eachexpected path line transmitted from the determination unit 1340. In thiscase, the superimposed video generation unit 1350 can superimpose theexpected path line determined to be superimposed on video data among theplurality of expected path lines in accordance with the informationtransmitted from the determination unit 1340. Specifically, thesuperimposed video generation unit 1350 may refrain from superimposingthe expected path line not determined to be superimposed on video databy the determination unit 1340 on the video data. The superimposed videogeneration unit 1350 transmits superimposed data, which is video datawhere the expected path line is superimposed on the video data, to thedisplay control unit 1360. Note that the determination unit 1340 may beincluded in the superimposed video generation unit 1350.

The display control unit 1360 receives the superimposed data transmittedfrom the superimposed video generation unit 1350. The display controlunit 1360 transmits the received superimposed data to the display unit220. The display control unit 1360 may transmit the superimposed data tothe display unit 220 or stop transmitting the superimposed data inresponse to an instruction from a display instruction unit, which is notshown, for example. The display control unit 1360 may monitor thevehicle motion detection unit 1300, and transmit the video data or thesuperimposed data to the display unit 220 or stop transmitting thosedata in accordance with information detected by the vehicle motiondetection unit 1300. Further, the display control unit 1360 may connectto the vehicle motion detection unit 1300 and acquire vehicle motioninformation.

The display control unit 1360 may process the received superimposed dataso as to change the viewing angle of a video image to be displayed onthe display unit. Further, the display control unit 1360 may process thereceived superimposed data so as to perform distortion correction of avideo image to be displayed on the display unit. Furthermore, thedisplay control unit 1360 may process the received superimposed data soas to change a video image to be displayed on the display unit to avideo image viewed from the top of the vehicle. The display control unit1360 may process the received superimposed data so as to change a videoimage to be displayed on the display unit in a way not limited theabove.

The camera 210 generates video data that shows a frontward or rearwardvideo of a vehicle. The camera 210 may be placed on the front of avehicle. Alternatively, the camera 210 may be placed on the back of avehicle. Further, the camera 210 may be placed on each of the front of avehicle and the back of the vehicle. The camera 210 transmits thegenerated video data to the video data acquisition unit 1320 in thedisplay control device 1200.

The display unit 220 displays the video data or superimposed datareceived from the display control unit 1360 in the display controldevice 1200. The display unit 220 is a display device such as a liquidcrystal display device, an organic EL (organic electro-luminescence)display device, or a head-up display, for example. The display unit 220is placed at a position that can be viewed by a driver while handling avehicle. The position that can be viewed by a driver while handling avehicle is a position on a center console, on a dashboard, on a meterpanel, on a rear-view mirror, near a steering wheel, on a windshield orthe like, for example.

The first expected path line is described hereinafter with reference toFIG. 10. FIG. 10 is a view showing an example of a video image where afirst expected path line is superimposed on video data in the displaycontrol system 1100 according to the fifth embodiment. The video image40 in FIG. 10 contains a first expected path line 50 in addition to thevideo image described in FIG. 3. The first expected path line 50 extendsalong a planned moving path in accordance with the steering angle from acenter part 11 along the width of the vehicle 10. In other words, thefirst expected path line 50 is line information indicating an expecteddriving path of the vehicle 10 in accordance with the steering angle ofthe vehicle 10. The first expected path line 50 is a curved line inaccordance with the steering angle of the vehicle 10 except when thevehicle 10 is moving in a straight line. The curvature of the firstexpected path line 50 is larger as the steering angle of the vehicle 10is larger.

The start point 50 s is contained in the center part 11 of the vehicle10. The center part 11 is a center region that occupies about one-thirdof the width of the vehicle 10 displayed on the video image 40. Thestart point 50 s may coincide with a center point 11 c in the centerpart 11. The end point 50 e is a point on a curved line extending fromthe start point 50 s along the planned moving path in accordance withthe steering angle of the vehicle.

The end point 50 e may indicate a position where the actual distancefrom the vehicle 10 corresponds to a predetermined value. In the videoimage 40, a length 50 y in the y-axis direction from the start point 50s to the end point 50 e corresponds to a distance of 3 meters, forexample, from the center part 11 of the vehicle 10. In this case, thelength 50 y up to the end point 50 e in the y-axis direction can befixed. Further, the position of the end point 50 e in the x-axisdirection varies depending on the steering angle of the vehicle 10. Asthe steering angle of the vehicle 10 is larger, the absolute value of alength 50 x in the x-axis direction increases. When the steeringdirection of the vehicle 10 is right in the video image 40, the firstexpected path line 50 extends from the center part 11 in the y-axispositive direction and the x-axis positive direction. Likewise, when thesteering direction of the vehicle 10 is left in the video image 40, thefirst expected path line 50 extends from the center part 11 in they-axis positive direction and the x-axis negative direction.

Note that, the length 50 y up to the end point 50 e in the y-axisdirection may remain unfixed, and it may change to draw a circle or anellipse around the start point 50 s. The length 50 y up to the end point50 e in the y-axis direction may remain unfixed, and it may change insuch a way that the length of the first expected path line 50 issubstantially the same even when the steering angle changes.

The first expected path line 50 is not necessarily a solid line as shownin FIGS. 10 and 10B, and it may be a dashed line or a dotted line. Thefirst expected path line 50 is not necessarily equally bold, and theirwidth may vary from the start point 50 s to the end point 50 e.

Further, the position of the center part 11 in this embodiment is acenter region that occupies about one-third of the width of the vehicle10 displayed on the video image 40. The start point 50 s may coincidewith the center point 11 c in the center part 11.

Note that the display form of superimposing the first expected path line50 on video data is referred to as a first display mode.

As described above, the first expected path line 50 extends in themoving direction of the vehicle 10 from the center part 11 of thevehicle 10. Thus, the possibility that the parking line and the firstexpected path line overlap is reduced when the driver moves the vehicleto the parking space.

A third expected path line is described hereinafter with reference toFIG. 11. FIG. 11 is a view showing an example of a video image where athird expected path line is superimposed on video data in the displaycontrol system 1100 according to the fifth embodiment. The video image40 in FIG. 11 contains a third expected path line 60, a third expectedpath line 61 and distance indicator lines 166 to 168, in addition to thevideo image described in FIG. 3. The third expected path line 60 extendsalong a planned moving path of a vehicle from a left end 12 along thewidth of the vehicle in the video image 40. The third expected path line61 extends along a planned moving path of a vehicle from a right end 13along the width of the vehicle in the video image 40. The third expectedpath lines 60 and 61 are a pair of line information extending along aplanned moving path of a vehicle from both ends of the vehicle in thewidth direction. The third expected path lines 60 and 61 may be lineinformation associated with the steering angle of the vehicle andindicating an expected driving path of the vehicle in accordance withthe steering angle of the vehicle. The length of the third expected pathlines 60 and 61 in the y-axis direction corresponds to a distance of 3meters, for example, from the both ends of the vehicle 10, just like thelength 50 y of the first expected path line 50 described in FIG. 10.

The distance indicator lines 166 to 168 are line information extendinghorizontally to connect the third expected path line 60 on the left andthe third expected path line 61 on the right in the video image 40. Thedistance indicator lines 166 to 168 indicate a predetermined distancefrom the end of the vehicle 10. The distance indicator line 166indicates a distance of 0.4 meter from the rear end of the vehicle 10,for example. The distance indicator line 167 indicates a distance of 1.0meter from the rear end of the vehicle 10, for example. The distanceindicator line 168 indicates a distance of 2.0 meters from the rear endof the vehicle 10, for example.

FIG. 11A shows the third expected path lines 60 and 61 in the statewhere the steering angle is 0° and the vehicle moves in a straight line.FIG. 11B shows the third expected path lines 60 and 61 in the statewhere the steering wheel of the vehicle 10 is controlled in the steeringdirection indicated by the arrow. By looking at video data on which thethird expected path line 60, the third expected path line 61 and thedistance indicator lines 166 to 168 are superimposed when moving thevehicle, the driver can grasp the distance between the vehicle'sexpected path line and a parking space 32 and the vehicle.

The determination unit 1340 can calculate the distance between the leftparking line 30 and the left third expected path line 60 contained inthe video image 40. To be specific, the determination unit 1340 sets areference point 62 and a reference point 63 along the third expectedpath line 60, for example. The vertical positions of the reference point62 and the reference point 63 can be predetermined positions. Thehorizontal positions of the reference point 62 and the reference point63 can vary depending on the position of the third expected path line60. Then, the determination unit 1340 calculates a distance D62 betweenthe reference point 62 and the parking line 30 in the horizontaldirection. Further, the determination unit 1340 calculates a distanceD63 between the reference point 63 and the parking line 30 in thehorizontal direction. Then, the determination unit 1340 can determinewhether the calculated distance D62 and distance D63 are within apredetermined range or not.

The case where the distance D62 and the distance D63 are within apredetermined range includes the case where the parking line 30 and thethird expected path line 60 are close to each other, the case where theparking line 30 and the third expected path line 60 are in contact witheach other, or the case where the parking line 30 and the third expectedpath line 60 intersect with each other. Note that each of apredetermined range for the distance D62 and a predetermined range forthe distance D63 is set appropriately, and they may be the same range ordifference ranges.

The determination unit 1340 calculates the distance between the rightparking line 31 and the right third expected path line 61 contained inthe video image 40, just like the example of the left side describedabove. To be specific, the determination unit 1340 sets a referencepoint 64 and a reference point 65 along the third expected path line 61,just like the case of calculating the distance between the left parkingline 30 and the left third expected path line 60. Then, thedetermination unit 1340 calculates a distance D64 between the referencepoint 64 and the parking line 31 in the horizontal direction. Further,the determination unit 1340 calculates a distance D65 between thereference point 65 and the parking line 31 in the horizontal direction.Then, the determination unit 1340 can determine whether the calculateddistance D64 and distance D65 are within a predetermined range or not.

The determination unit 1340 can determine the display form of the thirdexpected path lines 60 and 61 based on the distance informationcalculated in this manner. For example, the determination unit 1340 maydetermine to display the third expected path lines 60 and 61 in a lessvisible form when it determines that any of the distance between theparking line 30 and the third expected path line 60 and the distancebetween the parking line 31 and the third expected path line 61 isshorter than a predetermined distance. Further, the determination unit1340 may determine to display the first expected path line, which isdescribed with reference to FIG. 10, instead of the third expected pathlines 60 and 61, when it determines that any of the distance between theparking line 30 and the third expected path line 60 and the distancebetween the parking line 31 and the third expected path line 61 isshorter than a predetermined distance.

The third expected path lines 60 and 61 may be associated with thesteering angle of the vehicle. In this case, the distances D62 to D65vary depending on the steering angle of the vehicle. For example,comparing the distance D63 in FIG. 11A and the distance D63 in FIG. 11B,the distance D63 in FIG. 11B is longer than the distance D63 in FIG.11A. In this case, the steering angle is approximately 0° in FIG. 11A,and it changes to the right as indicated by the arrow in FIG. 11B. Thesteering angle in FIG. 11B is a steering angle Ad. Likewise, comparingthe distance D65 in FIG. 11A and the distance D65 in FIG. 11B, thedistance D65 in FIG. 11B is shorter than the distance D65 in FIG. 11A.

Note that the display form of superimposing the third expected pathlines 60 and 61 on video data is referred to as a second display mode.

In this configuration, a driver can adjust the steering angle of avehicle by looking at the third expected path lines 60 and 61 whenmoving the vehicle to a parking space. Further, a driver can adjust thestop position at the front or back of the vehicle by looking at thedistance indicator lines 166 to 168. However, the third expected pathlines 60 and 61 overlap a parking line in some cases. In such cases, itis difficult for a driver to find out the parking line in the displayedvideo image. Thus, there are cases where it is difficult for a driver tograsp the positional relationship between the vehicle and the parkingspace.

In the display control system 1100 according to the fifth embodiment,the determination unit 1340 determines to set the display form to thefirst display mode when the parking line 30 and 31 and the thirdexpected path line 60 and 61 are respectively displayed in closeproximity to each other.

For example, in the case where the vehicle 10 is moving with thesteering angle close to a straight line motion, the third expected pathline 60 and 61 and the parking line 30 and 31 displayed along the movingdirection of the vehicle 10 are likely to be respectively displayed inparallel. In this case, the parking line 30 and 31 and the thirdexpected path line 60 and 61 are likely to be respectively displayed inclose proximity to each other. Thus, the determination unit 1340determines the display form depending on whether the steering angle Adis smaller than a predetermined angle. When the steering angle Ad issmaller than a predetermined angle, the determination unit 1340determines to set the display form to the first display mode. The casewhere the steering angle Ad is smaller than a predetermined angle iswhen the steering angle Ad is smaller than 10°, for example.

A process of the display control device 1200 according to the fifthembodiment is described hereinafter with reference to FIG. 12. FIG. 12is a view showing a flowchart in the display control device 1200according to the fifth embodiment.

First, in the display control device 1200, the vehicle motion detectionunit 1300 detects information about the setting of the transmission ofthe vehicle 10 from a CAN or the like, which is an in-car network. Next,the display control unit 1360 acquires the information about the settingof the transmission of the vehicle 10 detected by the vehicle motiondetection unit 1300, and determines whether the vehicle 10 is to movebackward or not (Step S1100).

When the vehicle 10 is not to move backward, which is, when the settingof the transmission of the vehicle 10 is not a reverse range (No in StepS1100), the display control unit 1360 acquires information about thesetting of the transmission of the vehicle 10 detected by the vehiclemotion detection unit 1300 and performs processing of Step S1100 again.

On the other hand, when the vehicle 10 is moving backward, which is,when the setting of the transmission of the vehicle 10 is a reverserange (Yes in Step S1100), the display control device 1200 proceeds toStep S1101 to start processing of transmitting superimposed data to thedisplay unit 220.

Then, the determination unit 1340 acquires information about the settingof the transmission of the vehicle 10 detected by the vehicle motiondetection unit 1300, and determines whether the steering angle Ad issmaller than a predetermined angle A1 or not (Step S1101). When thedetermination unit 1340 determines that the steering angle Ad is smallerthan the predetermined angle A1 (Yes in Step S1101), the determinationunit 1340 transmits information indicating generation of superimposeddata in the first display mode to the superimposed video generation unit1350 (Step S1102). Therefore, the superimposed video generation unit1350 superimposes the video data transmitted from the expected path linegeneration unit 1310 on the video data transmitted from the video dataacquisition unit 1320 in the first display mode. The display controlunit 1360 transmits the superimposed data in the first display modetransmitted from the superimposed video generation unit 1350 to thedisplay unit 220.

On the other hand, when the determination unit 1340 does not determinethat the steering angle Ad is smaller than the predetermined angle A1(No in Step S1101), the determination unit 1340 transmits informationindicating generation of superimposed data in the second display mode tothe superimposed video generation unit 1350 (Step S1105). Therefore, thesuperimposed video generation unit 1350 superimposes the video datatransmitted from the expected path line generation unit 1310 on thevideo data transmitted from the video data acquisition unit 1320 in thesecond display mode. The display control unit 1360 transmits thesuperimposed data in the second display mode transmitted from thesuperimposed video generation unit 1350 to the display unit 220.

Then, after the determination unit 1340 determines the display form inStep S1102 or S1105, the display control unit 1360 determines whether ornot to end the display of the expected path line (S1103). To bespecific, the display control unit 1360 determines whether the settingof the transmission of the vehicle 10 is a reverse range or not bymonitoring the vehicle motion detection unit 1300, for example. When thesetting of the transmission of the vehicle 10 is not a reverse range(Yes in Step S1103), the display control device 1200 determines to endthe display of the expected path line. On the other hand, when thesetting of the transmission of the vehicle 10 is a reverse range (No inStep S1103), the display control device 1200 does not end the display ofthe expected path line. In this case, the process returns to Step S1101,and the determination unit 1340 acquires the information about thesetting of the transmission of the vehicle 10 detected by the vehiclemotion detection unit 1300, and continues processing.

By performing the above process, the display control system 1100according to the fifth embodiment can reduce difficulty of visuallyrecognizing the parking line displayed on the display unit.

Sixth Embodiment

A sixth embodiment is described hereinafter with reference to FIG. 13. Adisplay control system 1100 according to the sixth embodiment isdifferent from the display control system 1100 according to the fifthembodiment in that the determination unit 1340 makes determination basedon the distance between a parking line and the third expected path lines60 and 61.

FIG. 13 is a view showing an example of a video image in the displaycontrol system 1100 according to the sixth embodiment. In FIG. 13A, thethird expected path lines 60 and 61 and the distance indicator lines 166to 168 are superimposed on the video data acquired by the video dataacquisition unit 1320. The display form of the video image 40 shown inFIG. 13A is the second display mode.

The determination unit 1340 calculates the distance between the leftparking line 30 and the left third expected path line 60 in the videoimage 40, as described above with reference to FIG. 11. Likewise, thedetermination unit 1340 calculates the distance between the rightparking line 31 and the right third expected path line 61 in the videoimage 40. The determination unit 1340 thereby calculates each of thedistances D62 to D65.

In FIG. 13A, the distance D62 and the distance D63 are within apredetermined range. Stated differently, the parking line 30 and thethird expected path line 60 are in close proximity to each other. Insuch cases, it is difficult for a driver to find out the parking line 30in the video image 40. Thus, there are cases where it is difficult forthe driver to grasp the positional relationship between the vehicle andthe parking space. Therefore, when the distance between the thirdexpected path line 60 and 61 and the parking line 30 and 31 is within apredetermined range, the determination unit 1340 determines that theparking line 30 and 31 and the third expected path line 60 and 61 aredisplayed in close proximity to each other.

When the distance between the third expected path line 60 and 61 and theparking line 30 and 31 is within a predetermined range, thedetermination unit 1340 determines to set the display form of the videoimage 40 to the first display mode. In FIG. 13B, the first expected pathline 50 is superimposed on the video data acquired by the video dataacquisition unit 1320. The display form of the video image 40 shown inFIG. 13B is the first display mode.

The first expected path line 50 extends in the moving direction of thevehicle 10 from the center part 11 of the vehicle 10. Thus, thepossibility that the parking line and the first expected path lineoverlap is reduced when the driver moves the vehicle to the parkingspace. Thus, the display control system 1100 according to the sixthembodiment can reduce difficulty of visually recognizing the parkingline displayed on the display unit.

A process of the display control device 1200 according to the sixthembodiment is described hereinafter with reference to FIG. 14. FIG. 14is a view showing a flowchart in the display control device 1200according to the sixth embodiment. The flowchart of the display controldevice 1200 according to the sixth embodiment is different from theflowchart of the display control device 1200 according to the fifthembodiment in processing between Step S1100 and Step S1102.

When the vehicle 10 is moving backward, which is, when the setting ofthe transmission of the vehicle 10 is a reverse range (Yes in StepS1100), the display control device 1200 proceeds to Step S1201 to startprocessing of transmitting superimposed data to the display unit 220.

In Step S1201, the determination unit 1340 determines whether theparking line recognition unit 1330 recognizes a parking line or not(Step S1201). When the parking line recognition unit 1330 does notrecognize a parking line (No in Step S1201), the determination unit 1340transmits information indicating generation of superimposed data in thesecond display mode to the superimposed video generation unit 1350 (StepS1105).

On the other hand, when the parking line recognition unit 1330recognizes a parking line (Yes in Step S1201), the determination unit1340 determines whether the parking line and the third expected pathline are close to each other or not from the information about theparking line transmitted from the parking line recognition unit 1330 andthe information about the expected path line transmitted from theexpected path line generation unit 1310 (Step S1202). To be specific,the determination unit 1340 calculates the distances D62 to D65 asdescribed above with reference to FIG. 13, for example. Then, thedetermination unit 1340 determines whether the calculated distances D62to D65 are within a predetermined range or not.

When the determination unit 1340 does not determine that the parkingline and the third expected path line are close to each other (No inStep S1202), the determination unit 1340 transmits informationindicating generation of superimposed data in the second display mode tothe superimposed video generation unit 1350 (Step S1105).

When the determination unit 1340 determines that the parking line andthe third expected path line are close to each other (Yes in StepS1202), the determination unit 1340 transmits information indicatinggeneration of superimposed data in the first display mode to thesuperimposed video generation unit 1350 (Step S1102).

Then, after the determination unit 1340 determines the display form inStep S1102 or S1105, the display control unit 1360 determines whether ornot to end the display of the expected path line (S1103). To bespecific, the display control unit 1360 determines whether the settingof the transmission of the vehicle 10 is a reverse range or not bymonitoring the vehicle motion detection unit 1300, for example. When thesetting of the transmission of the vehicle 10 is not a reverse range(Yes in Step S1103), the display control device 1200 determines to endthe display of the expected path line. On the other hand, when thesetting of the transmission of the vehicle 10 is a reverse range (No inStep S1103), the display control device 1200 does not end the display ofthe expected path line. In this case, the process returns to Step S1101,and the determination unit 1340 acquires the information about thesetting of the transmission of the vehicle 10 detected by the vehiclemotion detection unit 1300, and continues processing.

By performing the above process, the display control system 1100according to the sixth embodiment can reduce difficulty of visuallyrecognizing the parking line displayed on the display unit.

Seventh Embodiment

A seventh embodiment is described hereinafter. A display control deviceaccording to the seventh embodiment further includes an object detectionsensor that detects an object existing around a vehicle a detection unitthat functions as a sensor detection unit.

The functional configuration of a display control system 1101 isdescribed hereinafter with reference to FIG. 15. FIG. 15 is a functionalblock diagram of the display control system 1101 according to theseventh embodiment. The display control system 1101 according to theseventh embodiment is different from the display control system 1100according to the fifth embodiment in that it has an object detectionsensor 230 and a sensor detection unit 1370. Further, it is differentfrom the display control system 1100 according to the fifth embodimentin the function of the determination unit 1340.

The display control system 1101 is a system that, when a driver ismoving a vehicle to a parking space, superimposes an expected path lineindicating a planned moving direction on a video image captured by acamera 210 that is mounted on the front or back of the vehicle and takesa video of a vehicle moving direction, and displays this video image ona display unit 220. In addition, the display control system 1101includes the object detection sensor 230, and further includes a displaycontrol device 1201 and at least one of the camera 210 and the displayunit 220.

The object detection sensor 230 is a sensor that detects a distance ofan object existing around a vehicle from the vehicle. The objectdetection sensor 230 is an ultrasonic sensor, an infrared sensor, alaser sensor, a millimeter-wave radar or the like, for example. Theobject detection sensor 230 may be a stereo camera. Further, a pluralityof object detection sensors 230 may be placed. The object detectionsensor 230 may be composed of a combination of them. The objectdetection sensor 230 is connected to the sensor detection unit 1370 inthe display control device 201.

The display control device 1201 is described hereinafter. The displaycontrol device 1201 is different from the display control device 1200according to the fifth embodiment in that it includes the sensordetection unit 1370 and a determination unit 1340 which is connected tothe sensor detection unit 1370, instead of the determination unit 1340.

The sensor detection unit 1370 detects information transmitted from theobject detection sensor 230. The sensor detection unit 1370 transmitsthe detected information to the determination unit 1340. The sensordetection unit 1370 can combine information transmitted from a pluralityof object detection sensors 230 and transmit the composite informationto the determination unit 1340, for example. Alternatively, the sensordetection unit 1370 may transmit information transmitted from aplurality of object detection sensors 230 to the determination unit 1340without combining them, for example. Alternatively, the sensor detectionunit 1370 may extract only predetermined information from informationtransmitted from a plurality of object detection sensors 230 andtransmit this information to the determination unit 1340, for example.The predetermined information may be information as to whether an objectexists within a predetermined distance from a vehicle, for example.Further, the predetermined information may be distance information froma vehicle to an object, for example.

The determination unit 1340 receives information transmitted from thevehicle motion detection unit 1300, information transmitted from theexpected path line generation unit 1310, information transmitted fromthe parking line recognition unit 1330, or information transmitted fromthe sensor detection unit 1370, and determines a display form of theexpected path line to be displayed on the display unit 220. Thedetermination unit 1340 transmits information about this determinationto the superimposed video generation unit 1350.

The determination unit 1340 can determine a display form of the expectedpath line based on the information transmitted from the vehicle motiondetection unit 1300 and the sensor detection unit 1370, for example.Further, the determination unit 1340 can calculate the distance betweenthe parking space and the expected path line from the information aboutthe expected path line transmitted from the expected path linegeneration unit 1310 and the information about the parking linetransmitted from the parking line recognition unit 1330, for example.Then, the determination unit 1340 can determine the display form of theexpected path line based on the calculation result and the informationtransmitted from the sensor detection unit 1370.

A vehicle on which the display control system 1101 is mounted isdescribed hereinafter with reference to FIG. 16. FIG. 16 is a top viewshowing an example of a state where a vehicle 10 on which the displaycontrol system 1101 is mounted is moving to a parking space 33. Thevehicle 10 includes the display control device 1201 at an arbitraryposition. The vehicle 10 also includes a forward camera 210F and arearward camera 210R. The vehicle 10 further includes the display unit220 at a position that can be viewed by a driver while handling thevehicle. In addition, the vehicle 10 includes the object detectionsensor 230 for detecting the presence or absence of an object around thevehicle 10. In FIG. 16, the vehicle 10 is moving backward toward theparking space 33. The parking space 33 includes parking lines 30 to 32.The driver is trying to move the vehicle 10 to a position surrounded bythe parking lines 30 to 32. In this case, the display control system1101 acquires data of a video captured by the rearward camera 210R.Further, another vehicle 15 exists in a parking space 34 adjacent to theparking space 33. Thus, the driver of the vehicle 10 moves the vehicle10 to the parking space 33 with a distance kept between the vehicle 10and another vehicle 15.

The display control system 1101 according to the seventh embodiment isfurther described with reference to FIG. 17. FIG. 17 is a view showingan example of a video image in the display control system 101 accordingto the seventh embodiment.

FIG. 17A is a view showing an example of a video image where a seventhexpected path line is superimposed on video data in the display controlsystem 1101 according to the third embodiment. In FIG. 17A, the vehicle10 is moving toward the parking space 33. The steering angle Ad issmaller than a predetermined angle. The vehicle 10 is moving with thesteering angle close to a straight line motion. Thus, the third expectedpath line 60 and 61 and the parking line 30 and 31 displayed along themoving direction of the vehicle 10 are likely to be displayed inparallel. In this case, the parking line 30 and 31 and the thirdexpected path line 60 and 61 are likely to be displayed in closeproximity to each other.

However, another vehicle 15 is parked in the parking space 34 adjacentto the parking space 33. In such a case, a driver needs to move thevehicle 10 to the parking space 33 with a distance kept from anothervehicle 15. Thus, the determination unit 1340 determines to set thedisplay form (second display mode) that superimposes the third expectedpath lines 60 and 61 regardless of the value of the steering angle Adwhen it detects a nearby object at a predetermined distance from thevehicle 10.

FIG. 17B is a view showing an example of a video image where a firstexpected path line is superimposed on video data in the display controlsystem 1101 according to the seventh embodiment. In FIG. 17B, anothervehicle 15 is not parked in the parking space 34. Thus, thedetermination unit 1340 determines a display form depending on whetherthe steering angle Ad is smaller than a predetermined angle when it doesnot detect a nearby object at a predetermined distance from the vehicle10. Specifically, the determination unit 1340 determines to set thedisplay form (first display mode) that superimposes the first expectedpath line when the steering angle Ad is smaller than a predeterminedangle.

In this configuration, the display control system 1101 according to theseventh embodiment can reduce difficulty of visually recognizing aparking line with a distance kept from a nearby object when a drivermoves a vehicle to a parking space.

A process of the display control device 1201 according to the seventhembodiment is described hereinafter with reference to FIG. 18. FIG. 18is a view showing a flowchart in the display control device 1201according to the seventh embodiment. The flowchart of the displaycontrol device 1201 according to the seventh embodiment is differentfrom the flowchart of the display control device 1200 according to thesixth embodiment in processing between Step S1202 and Step S1102.

When the determination unit 1340 determines that the parking line andthe third expected path line are close to each other in Step S1202 (Yesin Step S1202), the determination unit 1340 determines whether a nearbyobject around the vehicle 10 exists or not (Step S1301). Specifically,the determination unit 1340 determines whether a nearby object aroundthe vehicle 10 exists or not from the information transmitted from thesensor detection unit 1370.

When the determination unit 1340 determines that a nearby object aroundthe vehicle 10 exists (Yes in Step S1301), the determination unit 1340transmits information indicating generation of superimposed data in thesecond display mode to the superimposed video generation unit 1350 (StepS1105).

On the other hand, when the determination unit 1340 does not determinethat a nearby object around the vehicle 10 exists (No in Step S1301),the determination unit 1340 transmits information indicating generationof superimposed data in the first display mode to the superimposed videogeneration unit 1350 (Step S1102).

The subsequent process is the same as the process according to the firstand second embodiments described above. Note that, in the processaccording to the seventh embodiment, Step 1301 may be performed beforeStep 1201 or before Step S1202.

The display control system 1101 according to the seventh embodiment usesthe steering angle Ad in the determination as to whether the parkingline and the third expected path line are close to each other. However,the values of the distances D62 to D65 described in the sixth embodimentmay be used in the determination as to whether the parking line and thethird expected path line are close to each other.

By performing the above process, the display control system 1101according to the seventh embodiment can reduce difficulty of visuallyrecognizing the parking line displayed on the display unit.

Eighth Embodiment

An eighth embodiment is described hereinafter with reference to FIGS. 19and 20. A display control system 1101 according the eighth embodiment isthe same as the functional block diagram described with reference toFIG. 15. The display control system 1101 according the eighth embodimentis different from the display control system 1101 according to theseventh embodiment in that the determination unit 1340 calculates adistance between a parking line and a nearby object from informationtransmitted from the parking line recognition unit 1330 and informationtransmitted from the sensor detection unit 1370.

The eighth embodiment is described hereinafter in detail with referenceto FIG. 19. FIG. 19 is a view showing an example of a video image in thedisplay control system 1101 according to the eighth embodiment.

FIG. 19A is a view showing an example of a video image where a thirdexpected path line is superimposed on video data in the display controlsystem 1101 according to the eighth embodiment. In FIG. 19A, the vehicle10 is moving toward the parking space 33.

The determination unit 1340 calculates the distance between the leftparking line 30 and the left third expected path line 60 in the videoimage 40 in order to determine whether the parking line and the thirdexpected path line are close to each other or not. Likewise, thedetermination unit 1340 calculates the distance between the rightparking line 31 and the right third expected path line 61 in the videoimage 40. Specifically, the determination unit 1340 calculates each ofthe distances D62 to D65.

In FIG. 19A, the distance D62 and the distance D63 are within apredetermined range. Stated differently, the parking line 30 and thethird expected path line 60 are in close proximity to each other. Insuch cases, it is difficult for a driver to find out the parking line 30in the video image 40. Thus, there are cases where it is difficult forthe driver to grasp the positional relationship between the vehicle andthe parking space. When the distance between the third expected pathline 60 and 61 and the parking line 30 and 31 is within a predeterminedrange, the determination unit 1340 determines that the parking line 30and 31 and the third expected path line 60 and 61 are displayed in closeproximity to each other.

However, another vehicle 15 is parked in the parking space 34 adjacentto the parking space 33. In such a case, a driver needs to move thevehicle 10 to the parking space 33 with a distance kept from anothervehicle 15. Thus, the determination unit 1340 determines to set thedisplay form (second display mode) that superimposes the third expectedpath lines 60 and 61 regardless of the value of the distances D62 to D65when it detects a nearby object at a predetermined distance from theparking line 30 or 31. In FIG. 19A, a distance D30 is the distancebetween the parking line 30 and another vehicle 15. The determinationunit 1340 calculates the distance D30 from information about theposition and distance of another vehicle 15 detected by the objectdetection sensor 230 and information about the parking line 30recognized by the parking line recognition unit 1330.

When calculating the distance D30, the determination unit 1340 cananalyze the position and distance of another vehicle 15 and set a borderline of the detected object as indicated by a dashed line 15 b. Thedistance D30 may be the distance between the parking line 30 and anothervehicle 15 in the horizontal direction, for example. Further, thedistance D30 may be the shortest distance between the parking line 30and another vehicle 15.

FIG. 19B is a view showing an example of a video image where a firstexpected path line is superimposed on video data in the display controlsystem 1101 according to the eighth embodiment. In FIG. 19B, thedistance D30 between another vehicle 15 parked in the parking space 34and the parking line 30 is larger than a predetermined value. In thiscase, the determination unit 1340 determines that the parking line 30and 31 and the third expected path line 60 and 61 are displayed in closeproximity to each other when the distance between the third expectedpath line 60 and 61 and the parking line 30 and 31 is within apredetermined range. In other words, the determination unit 1340determines to set the display form (first display mode) thatsuperimposes the first expected path line.

In this configuration, the display control system 1101 according to theeighth embodiment can reduce difficulty of visually recognizing aparking line with a distance kept from a nearby object when a drivermoves a vehicle to a parking space.

A process of the display control device 1201 according to the eighthembodiment is described hereinafter with reference to FIG. 20. FIG. 20is a view showing a flowchart in the display control device 1201according to the eighth embodiment. The flowchart of the display controldevice 1201 according to the eighth embodiment is different from theflowchart of the display control device 1201 according to the seventhembodiment in processing between Step S1202 and

Step S1102.

When the determination unit 1340 determines that the parking line andthe third expected path line are close to each other in Step S1202 (Yesin Step S1202), the determination unit 1340 determines whether theparking line 30 and 31 and a nearby object around the vehicle 10 areclose to each other or not (Step S1401).

Specifically, the determination unit 1340 calculates the distance D30from information transmitted from the sensor detection unit 1370 andinformation transmitted from the parking line recognition unit 1330.Then, the determination unit 1340 determines whether the distance D30 issmaller than a predetermined distance D1 or not.

When the determination unit 1340 determines that the distance D30 issmaller than the distance D1 (Yes in Step S1401), the determination unit1340 transmits information indicating generation of superimposed data inthe second display mode to the superimposed video generation unit 1350(Step S1105).

On the other hand, when the determination unit 1340 does not determinethat the distance D30 is smaller than the distance D1 (No in StepS1401), the determination unit 1340 transmits information indicatinggeneration of superimposed data in the first display mode to thesuperimposed video generation unit 1350 (Step S1102).

The subsequent process is the same as the process according to the fifthto seventh embodiments described above. Note that, in the processaccording to the eighth embodiment, Step 1401 may be performed beforeStep 1201 or before Step S1202.

The display control system 1101 according to the eighth embodiment usesthe distances D62 to D65 in the determination as to whether the parkingline and the third expected path line are close to each other. However,the value of the steering angle Ad described in the fifth embodiment maybe used in the determination as to whether the parking line and thethird expected path line are close to each other.

By performing the above process, the display control system 1101according to the eighth embodiment can reduce difficulty of visuallyrecognizing a parking line displayed on a display unit.

Examples of Other Display Forms in First Display Mode

Examples of other display forms in the first display mode according tothe fifth to eighth embodiments are described hereinafter with referenceto FIGS. 21 to 23.

FIG. 21 is a view showing an example of a video image where a firstexpected path line and vehicle width guide lines are superimposed onvideo data. In the video image 40 shown in FIG. 21, vehicle width guidelines 76 to 78 are superimposed on the first expected path line 50described with reference to FIG. 10. The vehicle width guide lines 76 to78 are line information extending in the horizontal direction of thevideo image 40, which is the direction parallel to the x-axis. Thevehicle width guide lines 76 to 78 may indicate a predetermined distancefrom the end of the vehicle 10. The vehicle width guide line 76 mayindicate a distance of 0.4 meter from the rear end of the vehicle 10,for example. The vehicle width guide line 77 may indicate a distance of1.0 meter from the rear end of the vehicle 10, for example. The vehiclewidth guide line 78 may indicate a distance of 2.0 meters from the rearend of the vehicle 10, for example. In this case, the vehicle widthguide lines 76 to 78 may be lines that are not displaced in the y-axisdirection in the video image 40. The vehicle width guide lines 76 to 78may be the same as the distance indicator lines 166 to 168 describedwith reference to FIG. 11.

The length of the vehicle width guide lines 76 to 78 coincide with thewidth of the vehicle 10. For example, a left end 76L of the vehiclewidth guide line 76 indicates one end along the width of the vehicle 10at a distance of 0.4 meter from the rear end of the vehicle 10. On theother hand, a right end 76R of the vehicle width guide line 76 indicatesthe other end along the width of the vehicle 10 at a distance of 0.4meter from the rear end of the vehicle 10. Thus, the vehicle width guideline 76 is a straight line that extends along the width of the vehicle10 and coincides with the width of the vehicle 10 at a distance of 0.4meter from the rear end of the vehicle 10. Likewise, the vehicle widthguide line 77 is a straight line that extends along the width of thevehicle 10 and coincides with the width of the vehicle 10 at a distanceof 1.0 meter from the rear end of the vehicle 10. Further, the vehiclewidth guide line 78 is a straight line that extends along the width ofthe vehicle 10 and coincides with the width of the vehicle 10 at adistance of 2.0 meters from the rear end of the vehicle 10. The vehiclewidth guide lines 76 to 78 can be displaced in the x-axis direction inthe video image 40 in accordance with the steering angle of the vehicle10.

An example of another display form in the first display mode is furtherdescribed hereinafter with reference to FIG. 22. FIG. 22 is a viewshowing an example of a video image where a first expected path line anda second expected path line with high transparency are superimposed onvideo data. In the video image 40 shown in FIG. 22A, second expectedpath lines 52 and 53 are superimposed in addition to the first expectedpath line 50 described with reference to FIG. 4.

The second expected path lines 52 and 53 are a pair of line informationassociated with the steering angle of the vehicle 10. The secondexpected path lines 52 and 53 are line information indicating anexpected driving path of the vehicle 10 in accordance with the steeringangle of the vehicle 10. The second expected path line 52 extends alonga planned moving path in accordance with the steering angle from a leftend 12 along the width of the vehicle 10. Likewise, the second expectedpath line 53 extends along a planned moving path in accordance with thesteering angle from a right end 13 along the width of the vehicle 10.Thus, the second expected path lines 52 and 53 extend in the movingdirection of the vehicle 10 from the positions corresponding to thewidth of the vehicle 10. The second expected path lines 52 and 53 arecurved lines in accordance with the steering angle of the vehicle 10except when the vehicle 10 is moving in a straight line. The curvatureof the second expected path lines 52 and 53 is larger as the steeringangle of the vehicle 10 is larger.

In the example shown in FIG. 22A, the second expected path lines 52 and53 have predetermined transparency. The predetermined transparency is50% to 90%, for example. In a part where the second expected path lines52 and 53 overlap the video data in the superimposed data generated bythe superimposed video generation unit 1350, the second expected pathlines 52 and 53 are semi-transparent.

Further, the second expected path lines 52 and 53 have highertransparency than that of the first expected path line 50. For example,when the transparency of the second expected path lines 52 and 53 is50%, the transparency of the first expected path line 50 is 0%. Adifference between the transparency of the second expected path lines 52and 53 and the transparency of the first expected path line 50 ispreferably 50% or more.

In this configuration, the possibility that the parking line and thefirst expected path line overlap is reduced. Further, the displaycontrol device 1200 or the display control device 1201 superimposes thesecond expected path lines 52 and 53 and can thereby reduce difficultyof visually recognizing the parking line displayed on the display unitand also accurately display the positional relationship between thewidth of the vehicle 10 and the parking space.

FIG. 22B is described hereinafter. In the video image 40 shown in FIG.22B second expected path lines 54 and 55 are superimposed on the firstexpected path line 50.

The second expected path lines 54 and 55 are a pair of line informationcorresponding to the width of the vehicle 10 and extending from the endsalong the width of the vehicle 10 when the vehicle 10 moves in astraight line. The second expected path line 54 extends in the straightmoving direction of the vehicle 10 from a left end 12 along the width ofthe vehicle 10. Likewise, the second expected path line 55 extends inthe straight moving direction of the vehicle 10 from a right end 13along the width of the vehicle 10. Thus, the second expected path lines54 and 55 extend in the moving direction of the vehicle 10 from thepositions corresponding to the width of the vehicle 10. The secondexpected path lines 54 and 55 do not vary depending on the steeringangle of the vehicle 10. On the other hand, the first expected path line50 varies depending on the steering angle. Therefore, a distance 54 gbetween an end point 50 e of the first expected path line 50 and an endpoint 54 e of the second expected path line 54 varies depending on thesteering angle of the vehicle 10. Specifically, the distance 54 g islonger as the steering angle of the vehicle 10 is large to the right.Likewise, a distance 55 g between the end point 50 e of the firstexpected path line 50 and an end point 55 e of the second expected pathline 55 is longer as the steering angle of the vehicle 10 is large tothe left.

In this configuration, the possibility that the parking line and thefirst expected path line overlap is reduced. Further, the displaycontrol device 1200 according to the eighth embodiment superimposes thesecond expected path lines 54 and 55 and can thereby reduce difficultyof visually recognizing the parking line displayed on the display unitand also accurately display the positional relationship between thewidth of the vehicle 10 and the parking space and the information aboutthe steering angle.

An example of another display form in the first display mode is furtherdescribed hereinafter with reference to FIG. 23. FIG. 23 is a viewshowing a video image illustrating a variation of the first expectedpath line.

A first expected path line 51 superimposed on the video image 40 shownin FIG. 23 is a straight line. The first expected path line 51 extendslinearly along a planned moving path in accordance with the steeringangle from a start point 50 s coinciding with the center part 11 of thevehicle 10. Thus, the main expected path line 51 connects the startpoint 50 s and an end point 50 e by a straight line. The first expectedpath line 51 draws a straight line 51 y parallel to the y-axis when thevehicle 10 is moving in a straight line. An angle A51 between the firstexpected path line 51 and the straight line 51 y is larger as thesteering angle of the vehicle 10 is larger.

The start point 50 s is contained in the center part 11 of the vehicle10. The end point 50 e is a point on a curved line extending from thestart point 50 s along the planned moving path in accordance with thesteering angle of the vehicle 10.

In the examples of other display forms in the first display modedescribed above also, the possibility that the parking line and thefirst expected path line 50 or 51 overlap is reduced when the drivermoves the vehicle to the parking space. Further, the first expected pathline 50 or 51 is displayed in such a way that they can be accuratelyvisually recognized by a driver. The display control system 1100 and1101 according to the fifth to eighth embodiments with such displayforms can thereby reduce difficulty of visually recognizing the parkingline displayed on the display unit.

It should be noted that the above-described embodiments may be appliedto the case where the vehicle 10 moves forward to the parking space 33.In this case, the display control device 1200 can superimpose theexpected path line on data of a video captured by the forward camera210F of the vehicle 10. In this case, in Step S1100, the display controldevice 1200 can detect that the vehicle 10 is moving forward at a speedslower than a predetermined traveling speed, for example. Alternatively,in Step S1100, the display control device 1200 may start the firstdisplay mode in response to an instruction from a driver, which is notshown.

While the present invention has been particularly shown and describedwith reference to example embodiments thereof, the invention is notlimited to these example embodiments. It will be understood by those ofordinary skill in the art that various changes in form and details maybe made therein without departing from the spirit and scope of thepresent invention as defined by the claims.

A display control device or the like according to this embodiment can beused to superimpose predetermined information on a video image around avehicle in motion and display this video image, for example.

What is claimed is:
 1. A display control device comprising: a video dataacquisition unit configured to acquire video data from a camera thatcaptures a video image in a moving direction of a vehicle when thevehicle is moving toward a parking space where a parking line isdisplayed thereon; a vehicle motion detection unit configured to detectmotion information containing a steering angle of the vehicle; anexpected path line generation unit configured to generate a firstexpected path line extending along a planned moving path in accordancewith the steering angle from a center part along a width of the vehicleand a third expected path line including a pair of lines extending alonga planned moving path of the vehicle from both ends of the vehicle inthe width direction; a parking line recognition unit configured torecognize the parking line from the video data; a detection unitconfigured to detect a nearby object adjacent to the parking space; adetermination unit configured to determine to superimpose the firstexpected path line on the video data when the nearby object is notdetected by the detection unit and the parking line and the thirdexpected path line are to be displayed in close proximity to each other;a superimposed video generation unit configured to generate superimposeddata, the superimposed data being video data where the first expectedpath line or the third expected path line is superimposed on the videodata in accordance with a determination by the determination unit; and adisplay control unit configured to transmit the superimposed data to adisplay unit so as to display a video related to the superimposed dataon the display unit.
 2. The display control device according to claim 1,wherein the first expected path line generated by the expected path linegeneration unit is displayed on the display unit as a straight lineextending from the center part along the width of the vehicle.
 3. Thedisplay control device according to claim 2, wherein the expected pathline generation unit further generates a pair of second expected pathlines extending along a planned moving path of the vehicle from bothends of the vehicle in the width direction and having highertransparency than that of the first expected path line on the videodata.
 4. The display control device according to claim 3, wherein thesecond expected path line generated by the expected path line generationunit is displayed as a straight line extending in a straight movingdirection of the vehicle from both ends of the vehicle in the widthdirection.
 5. The display control device according to claim 1, whereinthe determination unit determines to superimpose the first expected pathline on the video data when the parking line and the third expected pathline are to be displayed in close proximity to each other.
 6. Thedisplay control device according to claim 5, wherein, when thedetermination unit determines to superimpose the first expected pathline on the video data, the superimposed video generation unit generatessuperimposed data, the superimposed data being video data where a secondexpected path line having higher transparency than that of the firstexpected path line is additionally superimposed.
 7. The display controldevice according to claim 6, wherein the second expected path lineincludes a pair of lines extending in a straight moving direction of thevehicle.
 8. The display control device according to claim 1, wherein thedetermination unit determines that the parking line and the thirdexpected path line are to be displayed in close proximity to each otherwhen the steering angle of the vehicle is smaller than a predeterminedangle.
 9. The display control device according to claim 1, furthercomprising: a parking line recognition unit configured to recognize aparking line from the video data, wherein the determination unitdetermines that the parking line and the third expected path line are tobe displayed in close proximity to each other when a distance betweenthe third expected path line and the parking line is within apredetermined range.
 10. The display control device according to claim1, wherein the expected path line generation unit further generates avehicle width guide line, the line being a line extending along thewidth of the vehicle and coinciding with the width of the vehicle in themoving direction.
 11. A display control system comprising: the displaycontrol device according to claim 1; and at least one of a cameraconfigured to capture the video data and a display unit configured todisplay the video data.
 12. A display control method comprising:acquiring video data from a camera that captures a video image in amoving direction of a vehicle when the vehicle is moving toward aparking space where a parking line is displayed thereon; detectingmotion information containing a steering angle of the vehicle;generating a first expected path line extending along a planned movingpath in accordance with the steering angle from a center part along awidth of the vehicle and a third expected path line including a pair oflines extending along a planned moving path of the vehicle from bothends of the vehicle in the width direction; recognizing the parking linefrom the video data; detecting a nearby object adjacent to the parkingspace; determining to superimpose the first expected path line on thevideo data when the nearby object is not detected by the detection unitand the parking line and the third expected path line are to bedisplayed in close proximity to each other generating superimposed data,the superimposed data being video data where the first expected pathline or the third expected path line is superimposed on the video datain accordance with a determination; and transmitting the superimposeddata to a display unit so as to display a video related to thesuperimposed data on the display unit.
 13. A computer readablenon-transitory storage medium storing a display control program causinga computer to execute: acquiring video data from a camera that capturesa video image in a moving direction of a vehicle when the vehicle ismoving toward a parking space where a parking line is displayed thereon;detecting motion information containing a steering angle of the vehicle;generating a first expected path line extending along a planned movingpath in accordance with the steering angle from a center part along awidth of the vehicle and a third expected path line including a pair oflines extending along a planned moving path of the vehicle from bothends of the vehicle in the width direction; recognizing the parking linefrom the video data; detecting a nearby object adjacent to the parkingspace; determining to superimpose the first expected path line on thevideo data when the nearby object is not detected by the detection unitand the parking line and the third expected path line are to bedisplayed in close proximity to each other; generating superimposeddata, the superimposed data being video data where the first expectedpath line or the third expected path line is superimposed on the videodata in accordance with a determination; and transmitting thesuperimposed data to a display unit so as to display a video related tothe superimposed data on the display unit.